DataMasque AWS blueprint templates are written in AWS CloudFormation format. The purpose of this template is to create a reusable data provisioning pipeline that calls DataMasque APIs to produce masked data that's safe for consumption in non-production environment.

The diagram below describes the DataMasque reference architecture in AWS. This CloudFormation template is used to mask production Aurora snapshots, create masked Aurora snapshots which can be used to provision non-production databases - this incorporates the automation steps highlighted in blue. Please refer to DataMasque AWS Service Catalog Template to use AWS Service Catalog product as an End User Interface to provision non-production databases using masked RDS Aurora snapshots.

The following lists the main AWS resources provisioned when this CloudFormation template is deployed:

• An AWS stepfunction.
• Five (5) AWS Lambda functions.
• A SQS queue.
• A CloudWatch event rule to schedule the Step Function execution once a week.

The provisioned stepfunctions coordinates tasks by calling AWS components/services and DataMasque masking APIs to irreversibly replaces sensitive data such as PII, PCI and PHI with realistic, functional and consistent values.

You can trigger a data masking workflow by providing an RDS Aurora identifier to invoke execution of the deployed stepfunctions. Upon completing the data masking workflow, an encrypted and masked RDS Aurora snapshot is produced, ready to be used to provision non-production databases.

The diagram below describes the connectivities between the DataMasque instance, AWS Lambda functions (provisioned by this template) and the staging RDS Aurora Cluster (provisioned by this template).

• AWS CLI configured with appropriate credential for the target AWS account.
• AWS SAM CLI: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html
• Python runtime 3.9 installed.
• A DataMasque instance.
• Source database with automated snapshots enabled.
• DataMasque connection id and ruleset id.

Make sure you have created a secret with the format below:

{"username":"datamasque","password":"Example$P@ssword"}

1. Clone this repository
2. Open a terminal in the cloned repository directory
3. Run sam build.
4. Run sam deploy --guided.

During the guided deployment, you will be asked if you would like to save the parameters in an AWS SAM configuration file samconfig.toml.

An example of the configuration file is presented below:

version = 0.1
[default]
[default.deploy]
[default.deploy.parameters]
stack_name = "datamasque-blueprint-aurora"
s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-xxxxxxxxx"
s3_prefix = "datamasque-blueprint"
region = "ap-southeast-2"
capabilities = "CAPABILITY_IAM"
image_repositories = []

parameter_overrides = "VpcId=\"vpc-xxxxxxxx\" SubnetIds=\"subnet-xxxxxxxxxxxxx\" DatamasqueBaseUrl=\"https://10.11.12.13/\" DatamasqueSecretArn=\"xxxxxxxx\"  DatamasqueConnectionId=\"41xxxxxe7-fxxd-xxx5-8xxe-62xxxxxxxxe62\" DatamasqueRuleSetId=\"2bxxxxxxxb-3xxc-4xxe-axxxb-c35xxxxxxxx40\""

• The source RDS Aurora DB Cluster must allow inbound connections from the DataMasque EC2 instance. The configuration will be replicated when creating the staging RDS Aurora.
• The DataMasque EC2 instance must allow inbound connections from the DatamasqueRun Lambda.
• The DataMasque EC2 instance must allow inbound connections from the WaitDatamasqueRun Lambda.
• Grant permissions for the stepfunctions and lambda functions to use the KMS key configured on the source database to encrypt masked snapshots if you are not using the default RDS key. Note: this template assumes the source database uses default RDS KMS keyas every organization might have different key configuration standard.

You can also execute the step function manually.

{
  "DBClusterIdentifier": "source-postgres-rds"
}

The AWS SAM template creates a CloudWatch event rule that schedules a Step Function execution once a week which is disabled by default. To use this scheduling functionality, you will need edit the rule to specify your target DBClusterIdentifier and enable the CloudWatch event rule.

Events:
  Schedule:
    Type: Schedule # More info about Schedule Event Source: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-statemachine-schedule.html
    Properties:
      Description: Schedule to run the DATAMASQUE state machine weekly
      Enabled: False
      Schedule: "rate(7 days)"
      Input: '{"DBClusterIdentifier": "source-postgres-rds"}'

• The staging RDS Aurora Cluster created will follow the same RDS Aurora endpoint name schema as the source database with a -datamasque postfix after the DBInstanceIdentifier:

• The RDS Aurora username, password and connection port will be the same as the source RDS Aurora instance.

• The staging RDS Aurora instance created during the execution of the stepfunction will be deleted when the execution is completed.

• The masked RDS Aurora snapshot created during the execution of the stepfunction will be preserved when the execution is completed.

The following table describes the states and details of the step function definition.

Sharing snapshots encrypted with the default service key for RDS Aurora is currently not supported. Sharing a DB snapshot.

To share your encrypted snapshot with another account, you will also need to share the custom master key with the other account through KMS. Changing a key policy .

The masked snapshot can be shared with the following methods:

• Add an RDS Aurora modify db snapshot step to the lambda function.
• Use the native mechanism within the AWS Console.
• Use an existing CI/CD pipeline to copy and re-encrypt the snapshot.

• Create the DataMasque connection with the staging RDS Aurora instance dynamically.